<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
                "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
  <title>Description of demolgd1</title>
  <meta name="keywords" content="demolgd1">
  <meta name="description" content="DEMOLGD1 Demonstrate simple MLP optimisation with on-line gradient descent">
  <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
  <meta name="generator" content="m2html &copy; 2003 Guillaume Flandin">
  <meta name="robots" content="index, follow">
  <link type="text/css" rel="stylesheet" href="../../m2html.css">
</head>
<body>
<a name="_top"></a>
<div><a href="../../menu.html">Home</a> &gt;  <a href="#">ReBEL-0.2.7</a> &gt; <a href="#">netlab</a> &gt; demolgd1.m</div>

<!--<table width="100%"><tr><td align="left"><a href="../../menu.html"><img alt="<" border="0" src="../../left.png">&nbsp;Master index</a></td>
<td align="right"><a href="menu.html">Index for .\ReBEL-0.2.7\netlab&nbsp;<img alt=">" border="0" src="../../right.png"></a></td></tr></table>-->

<h1>demolgd1
</h1>

<h2><a name="_name"></a>PURPOSE <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="box"><strong>DEMOLGD1 Demonstrate simple MLP optimisation with on-line gradient descent</strong></div>

<h2><a name="_synopsis"></a>SYNOPSIS <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="box"><strong>This is a script file. </strong></div>

<h2><a name="_description"></a>DESCRIPTION <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="fragment"><pre class="comment">DEMOLGD1 Demonstrate simple MLP optimisation with on-line gradient descent

    Description
    The problem consists of one input variable X and one target variable
    T with data generated by sampling X at equal intervals and then
    generating target data by computing SIN(2*PI*X) and adding Gaussian
    noise. A 2-layer network with linear outputs is trained by minimizing
    a  sum-of-squares error function using on-line gradient descent.

    See also
    <a href="demmlp1.html" class="code" title="">DEMMLP1</a>, <a href="olgd.html" class="code" title="function [net, options, errlog, pointlog] = olgd(net, options, x, t)">OLGD</a></pre></div>

<!-- crossreference -->
<h2><a name="_cross"></a>CROSS-REFERENCE INFORMATION <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
This function calls:
<ul style="list-style-image:url(../../matlabicon.gif)">
<li><a href="mlp.html" class="code" title="function net = mlp(nin, nhidden, nout, outfunc, prior, beta)">mlp</a>	MLP	Create a 2-layer feedforward network.</li><li><a href="mlpfwd.html" class="code" title="function [y, z, a] = mlpfwd(net, x)">mlpfwd</a>	MLPFWD	Forward propagation through 2-layer network.</li><li><a href="mlpinit.html" class="code" title="function net = mlpinit(net, prior)">mlpinit</a>	MLPINIT Initialise the weights in a 2-layer feedforward network.</li><li><a href="olgd.html" class="code" title="function [net, options, errlog, pointlog] = olgd(net, options, x, t)">olgd</a>	OLGD	On-line gradient descent optimization.</li></ul>
This function is called by:
<ul style="list-style-image:url(../../matlabicon.gif)">
<li><a href="demnlab.html" class="code" title="function demnlab(action);">demnlab</a>	DEMNLAB A front-end Graphical User Interface to the demos</li></ul>
<!-- crossreference -->


<h2><a name="_source"></a>SOURCE CODE <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="fragment"><pre>0001 <span class="comment">%DEMOLGD1 Demonstrate simple MLP optimisation with on-line gradient descent</span>
0002 <span class="comment">%</span>
0003 <span class="comment">%    Description</span>
0004 <span class="comment">%    The problem consists of one input variable X and one target variable</span>
0005 <span class="comment">%    T with data generated by sampling X at equal intervals and then</span>
0006 <span class="comment">%    generating target data by computing SIN(2*PI*X) and adding Gaussian</span>
0007 <span class="comment">%    noise. A 2-layer network with linear outputs is trained by minimizing</span>
0008 <span class="comment">%    a  sum-of-squares error function using on-line gradient descent.</span>
0009 <span class="comment">%</span>
0010 <span class="comment">%    See also</span>
0011 <span class="comment">%    DEMMLP1, OLGD</span>
0012 <span class="comment">%</span>
0013 
0014 <span class="comment">%    Copyright (c) Ian T Nabney (1996-2001)</span>
0015 
0016 
0017 <span class="comment">% Generate the matrix of inputs x and targets t.</span>
0018 
0019 ndata = 20;            <span class="comment">% Number of data points.</span>
0020 noise = 0.2;            <span class="comment">% Standard deviation of noise distribution.</span>
0021 x = [0:1/(ndata - 1):1]';
0022 randn(<span class="string">'state'</span>, 42);
0023 rand(<span class="string">'state'</span>, 42);
0024 t = sin(2*pi*x) + noise*randn(ndata, 1);
0025 
0026 clc
0027 disp(<span class="string">'This demonstration illustrates the use of the on-line gradient'</span>)
0028 disp(<span class="string">'descent algorithm to train a Multi-Layer Perceptron network for'</span>)
0029 disp(<span class="string">'regression problems.  It is intended to illustrate the drawbacks'</span>)
0030 disp(<span class="string">'of this algorithm compared to more powerful non-linear optimisation'</span>)
0031 disp(<span class="string">'algorithms, such as conjugate gradients.'</span>)
0032 disp(<span class="string">' '</span>)
0033 disp(<span class="string">'First we generate the data from a noisy sine function and construct'</span>)
0034 disp(<span class="string">'the network.'</span>)
0035 disp(<span class="string">' '</span>)
0036 disp(<span class="string">'Press any key to continue.'</span>)
0037 pause
0038 <span class="comment">% Set up network parameters.</span>
0039 nin = 1;            <span class="comment">% Number of inputs.</span>
0040 nhidden = 3;            <span class="comment">% Number of hidden units.</span>
0041 nout = 1;            <span class="comment">% Number of outputs.</span>
0042 alpha = 0.01;            <span class="comment">% Coefficient of weight-decay prior.</span>
0043 
0044 <span class="comment">% Create and initialize network weight vector.</span>
0045 net = <a href="mlp.html" class="code" title="function net = mlp(nin, nhidden, nout, outfunc, prior, beta)">mlp</a>(nin, nhidden, nout, <span class="string">'linear'</span>);
0046 <span class="comment">% Initialise weights reasonably close to 0</span>
0047 net = <a href="mlpinit.html" class="code" title="function net = mlpinit(net, prior)">mlpinit</a>(net, 10);
0048 
0049 <span class="comment">% Set up vector of options for the optimiser.</span>
0050 options = foptions;
0051 options(1) = 1;            <span class="comment">% This provides display of error values.</span>
0052 options(14) = 20;        <span class="comment">% Number of training cycles.</span>
0053 options(18) = 0.1;        <span class="comment">% Learning rate</span>
0054 <span class="comment">%options(17) = 0.4;        % Momentum</span>
0055 options(17) = 0.4;        <span class="comment">% Momentum</span>
0056 options(5) = 1;         <span class="comment">% Do randomise pattern order</span>
0057 clc
0058 disp(<span class="string">'Then we set the options for the training algorithm.'</span>)
0059 disp([<span class="string">'In the first phase of training, which lasts for '</span>,<span class="keyword">...</span>
0060     num2str(options(14)), <span class="string">' cycles,'</span>])
0061 disp([<span class="string">'the learning rate is '</span>, num2str(options(18)), <span class="keyword">...</span>
0062     <span class="string">' and the momentum is '</span>, num2str(options(17)), <span class="string">'.'</span>])
0063 disp(<span class="string">'The error values are displayed at the end of each pass through the'</span>)
0064 disp(<span class="string">'entire pattern set.'</span>)
0065 disp(<span class="string">' '</span>)
0066 disp(<span class="string">'Press any key to continue.'</span>)
0067 pause
0068 
0069 <span class="comment">% Train using online gradient descent</span>
0070 [net, options] = <a href="olgd.html" class="code" title="function [net, options, errlog, pointlog] = olgd(net, options, x, t)">olgd</a>(net, options, x, t);
0071 
0072 <span class="comment">% Now allow learning rate to decay and remove momentum</span>
0073 options(2) = 0;
0074 options(3) = 0;
0075 options(17) = 0.4;    <span class="comment">% Turn off momentum</span>
0076 options(5) = 1;        <span class="comment">% Randomise pattern order</span>
0077 options(6) = 1;        <span class="comment">% Set learning rate decay on</span>
0078 options(14) = 200;
0079 options(18) = 0.1;    <span class="comment">% Initial learning rate</span>
0080 
0081 disp([<span class="string">'In the second phase of training, which lasts for up to '</span>,<span class="keyword">...</span>
0082     num2str(options(14)), <span class="string">' cycles,'</span>])
0083 disp([<span class="string">'the learning rate starts at '</span>, num2str(options(18)), <span class="keyword">...</span>
0084     <span class="string">', decaying at 1/t and the momentum is '</span>, num2str(options(17)), <span class="string">'.'</span>])
0085 disp(<span class="string">' '</span>)
0086 disp(<span class="string">'Press any key to continue.'</span>)
0087 pause
0088 [net, options] = <a href="olgd.html" class="code" title="function [net, options, errlog, pointlog] = olgd(net, options, x, t)">olgd</a>(net, options, x, t);
0089 
0090 clc
0091 disp(<span class="string">'Now we plot the data, underlying function, and network outputs'</span>)
0092 disp(<span class="string">'on a single graph to compare the results.'</span>)
0093 disp(<span class="string">' '</span>)
0094 disp(<span class="string">'Press any key to continue.'</span>)
0095 pause
0096 
0097 <span class="comment">% Plot the data, the original function, and the trained network function.</span>
0098 plotvals = [0:0.01:1]';
0099 y = <a href="mlpfwd.html" class="code" title="function [y, z, a] = mlpfwd(net, x)">mlpfwd</a>(net, plotvals);
0100 fh1 = figure;
0101 plot(x, t, <span class="string">'ob'</span>)
0102 hold on
0103 axis([0 1 -1.5 1.5])
0104 fplot(<span class="string">'sin(2*pi*x)'</span>, [0 1], <span class="string">'--g'</span>)
0105 plot(plotvals, y, <span class="string">'-r'</span>)
0106 legend(<span class="string">'data'</span>, <span class="string">'function'</span>, <span class="string">'network'</span>);
0107 hold off
0108 
0109 disp(<span class="string">'Note the very poor fit to the data: this should be compared with'</span>)
0110 disp(<span class="string">'the results obtained in demmlp1.'</span>)
0111 disp(<span class="string">' '</span>)
0112 disp(<span class="string">'Press any key to exit.'</span>)
0113 pause
0114 close(fh1);
0115 clear all;</pre></div>
<hr><address>Generated on Tue 26-Sep-2006 10:36:21 by <strong><a href="http://www.artefact.tk/software/matlab/m2html/">m2html</a></strong> &copy; 2003</address>
</body>
</html>